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ABSTRACT 

Eight surrogate spaceflight mission specialists participated in a real-time evaluation of 
remote coaching using the Ames Life Science Telescience Testbed facility. This facility con- 
sisted of three remotely located "nodes": (1) a prototype Space Station glovebox, (2) a ground 
^control station. and (3) a principal investigators ,Pr„ work area. THe major t ***** 
this project was to evaluate the effectiveness of telescience techniques and hardware to sup 
port three realistic remote coaching science procedures: plant seed germinator 
plant sample acquisition and preservation, and remote plant observation with ground coach- 
ins Each scenario was performed by a subject acting as flight mission specialist interacting 
wlh a payloZ operation! manager and a principal investigator expert All three groups 
were physically isolated from each other yet linked by duplex audio and color video communt- 
catio! channels and networked computer workstations. Workload ratings were made _by the 
flisht and ground crewpersons immediately after completing their assigned tasks. Time to 
Complete each scientific procedural step was recorded automatically. Two expert observers 
also made performance ratings and various error assessments. The results indicated that. (1) 
The Zrall quality of science performed increased when audio, video, and computer worksta- 
tions were available, (2) Loss of video resulted in many errors that were not caught by the 
PI or ground controller. Most would have resulted in loss of the nssue sample. Ot ers w 
Zve resulted in safety hazards and/or injury, (3) Once the subject was trained in a P r °“ d “ re 
they tended to use the computer-derived procedure checklist more as a backup onl y> (4) 
audio channel was considered to be the single most crucial communication means of the 
three tested. (5) Physical workload is significantly reduced during runs when m-o-vva.v colo 
video is available than when it is not. Mental workload is not so influenced. ) P 

prototyping approach to hardware, software, and procedure validation is sound and cost ff 

rive. 
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INTRODUCTION 


Telescience is an operational mode which enables the effective conduct of science through 
the use of remote resources including other people. Telescience may be subdivided into at 
least three principal components: teledesign, teleanalysis, and teleoperations. The Ames 
Life Science Telescience Testbed was designed as a rapid prototyping facility to evaluate 
teleoperations. Teleoperations is defined as: 

Operation of a laboratory remotely by providing capabilities to allow 
full user interaction as if the instrumentation were physically present. 

In addition, teleoperations enables the conduct of intra-and inter-active 
science where immediate scientific results can be evaluated by the inves- 
tigator in real time or new experiments can be carried out as new pheno- 
mena are uncovered. 

Effectively planned and executed teleoperations should allow a scientist at his or her 
home institution to carry out an experiment remotely using realtime displays of all relevant 
scientific and telemetry data. Reliable interactive control of the instruments also should be 
possible. Of course certain limits may be imposed by safety, security, and resource con- 
straints. 

Telescience is valuable in carrying out life science experiments in space for a number of 
reasons. For one, the nature of this research differs from that in the physical sciences be- 
cause the flight crew may serve both as experimenter and test subject. This fact calls for 
carefully planned test apparatus that can be monitored from the ground to ensure that the 
crew are well and the data is of adequate quality. For another, unexpected phenomena may 
occur which were not anticipated; the investigator may need to make real-time decisions to 
alter test conditions as the data is being collected. The application of properly designed tele- 
science means of connectivity makes this possible. Life science experiments may also call for 
(Treater flexibility of planning and conducting the investigations than do physical science ex- 
periments In addition, many life science procedures are time consuming and repetitive; re- 
mote semi-autonomous robotic mechanisms under telecontrol may be used to free up crew- 
time The large computational power of computers, linking earth with the space station s 
telescience communication channels, also makes it possible to carry out intense, elegant, cre- 
ative, and responsive ground analysis of realtime data. 

Test Objectives: 

In order to ensure that the variety of claimed capabilities of teleoperations actually are 
achievable it is essential to carry out manned tests with representative, working hardware 
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and software. This evaluation had four primary objectives: (1) Demonstrate savings in 

crew time flexibility of scheduling, enhanced productivity, and quality of science, if possible, 
within a semi-realistic test environment representing Space Station, (2) Evaluate interac- 
tional telescience modes using various combinations of flight crew, ground controller ground 
expert, and a low fidelity autonomous system, (3) Measure crew performance and system 
impacts and means of coping with off-nominal events, and (4) Provide the capability for 
audio-visual teleconferencing, information transfer, and transaction management. 


In this evaluation a number of interactions between representative operational elements 
and all participants were evaluated. The operational elements included three remotely locat- 
ed networked workstations and their collaborative software, duplex audio communication 
system, two-way closed circuit color TV networks, and a telerobotically controlled robot lo- 
cated inside the glovebox. The participants included a number of surrogate mission special- 
ists, ground controller and ground scientist/expert. Details of the system’s architecture are 
given elsewhere (Johnson, in preparation). 


METHOD 


Experimental Design: , , 

The multifaceted objectives of this investigation called for a vanety of participant group 

and dependent measures. There were three participant groups and four quantitative mea- 
sures of interest. The first group of participants was the eight test subjects who provided the 
following behavioral response measures: subjective workload estimates following each sce- 
nario time to carry out each step in each scenario, and general comments and observations 
about the design of the glovebox, procedures, and any other subject of interest (immediately 
following completion of the test). The second participant group consisted of the ground con- 
troller (author RFH) located at the Payload Operations Control Center (POCC) and the prin- 
cipal investigator (author KV) located at the Pi’s station. Each provided subjective work- 
load estimates following each scenario at the same time these estimates were given by the 
test subject, and general observations concerning the functioning of the hardware and the ad- 
equacy of the procedures. These comments generally were given after the conclusion of the 
study while some were offered during the study. The third participant group was the two ex- 
pert raters (authors VJ, WF) who kept notes throughout the data collection penod on the 

items listed in Appendix A. 

Each subject carried out three life science glovebox science procedures (also called sce- 
narios) a number of times during this investigation. One of the test approaches used was to 
ensure that each subject had reached performance asymptote during all three scenarios and 
then note changes in performance caused by the occurrence of an unanticipated failure. Perfor- 
mance during ^asymptotic response periods would also provide valuable insight about how 
the available” telescience modes are used. Four types of failures were presented to evaluate 
the usefulness of the available telescience modes as described below. A second feamre of 
the experimental design was to train half of the participants to operate with and the other 
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half without any computer networked support and then compare their performance. A third 
testing approach was to have half of the participants carry out the three scenarios without 
any video link so that their performance could be compared with the other half that did have 

two-way video. 

To demonstrate anticipated advantages of these telescience technologies on crew time 
savings, scheduling flexibility, enhanced productivity, and science quality, a team of expert 
observers were asked to make periodic human and system perfoimance judgments using the 
form shown in Appendix A. Analysis of video recordings, performance error analysis, and 
workload ratings were used to evaluate the effectiveness of the various telescience technolo- 
gies employed (viz., computer-based prompting, two-way voice and video communications, 
pre-programmed telerobotic operations, and networked workstations with electronic mad). 
Insertion of an unanticipated system failure for each subject made it possible to evaluate the 
use of telescience technologies for coping with off-nominal events. The three specific science 
scenarios used were chosen to evaluate the utility of the teleconferencing and workstation 
collaboration capability as well as the effectiveness of information transfer. 

Workload Rating. Using a standardized response menu which was available at their 
workstation, each subject, ground controller, and PI completed a workload rating senes im- 
mediately following each scenario. As shown in Figure 1, which is a reproduction of the ma- 
cintosh screen image presented, three workload measures were quantified: physical, mental, 
and total. The physical effort scale represented the subjectively determined amount of phys- 
ical effort that had to be expended to accomplish all of the required tasks in a scenario. Like- 
wise the mental effort scale referred to the amount of mental effort, cognitive effort or con- 
centration that had to be expended to accomplish all of the required tasks in a scenario. The 
total effort rating incorporated all of the contributors which make up one’s own workload es- 
timate (e.g., emotional response, intellectual involvement, psychological response mental 
and physical workload). Total workload is not merely the sum of the physical and mental 

workload estimates. 

Referring to Figure 1, when the subject clicked and held the mouse input to hold the cursor 
on the physical effort box a menu with seven numbered responses appeared: none, very low, 
low, medium, medium high, high, very high. When the mouse was dragged down tins scale, 
stopped on the desired rating, and then released, that answer was input to a file for subse- 
quent printout and analysis. These responses were available for inspection immediately af- 
ter each group of trials, which proved to be a useful capability. 

Expert Ratings. Each of the two ground experts used a special form to comment on and 
rate a variety of topics (cf. Appendix A). This was done during each scenario. This form 
contained the following issues and concerns: (1) Evidence for savings in crew time, schedu - 
ing flexibility improvements, productivity enhancement, science quality enhancement, and 
safetv enhancement. (2) Demonstration^ ) of workstation teleconferencing capability, video 
teleconferencing, audio teleconferencing, and effective information transfer. (3) General com- 
ments on space to ground, ground to space, ground to ground telescience interactions, and 
robotic operations. (4) Number and type of errors committed. (5) Unanticipated problems. 


RIACSTR 89.31 



Telescience Testbed 


Haines, et al. 


Page 4 


Figure 1 

Work Load Assessment Screen 


Help 


Communications 


Science Procedures Return 


2:54 AM 


Work Load Assessment 

The Scenario has been completed. Please click on the 
three workload scales given below and assess the degree 
of effort you had to expend to accomplish all tasks In 
the scenario. 


Physical Effort 
Mental Effort 
Total Effort 


Click here when assessment is completed: 
f Completed ) 


Regarding the assessment of errors, each expert kept track of the number and kinds of er- 
rors that were committed during each scenario. An error was defined in relation to "nominal 
or expected" performance by each subject on each scenario. The expen observed each sub- 
ject’s activities (remotely) throughout the familiarization/training runs in order to develop a 
general concept of what was considered "nominal or expected" performance during later data 
runs. The test subject, ground controller, and PI also made written note(s) of unanticipated 
problems which occurred. 

Task Accomplishment Time. A computer monitored the amount of time required by the 
subject to finish each step during each scenario. These times, in seconds, were stored and 
then printed out following each scenario for later analysis. 

Apparatus: 

The test hardware used can best be described in terms of the three remotely located labo- 
ratory areas and the hardware each contained: Control, Monitoring Area. (jloxebox Ar- 
ea. Ground Expert Area. Figure 2 is a diagram of all three areas approximately to scale 
showing the various video components used in this test. Figure .> is a schemat c diagram of 
all of the audio communication hardware that was used. 
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1. ControlIMonitoring Area. The ground control station was located in a floor area ap- 
proximately 8 by 10 feet which is shown in Figure 4. A large screen (29") high resolution 
color TV monitor (left) was used to display the output from any of the six closed circuit cam- 
eras to the ground controller. Cameras 1 - 3 were situated inside the glovebox (as will be 
described) while camera 4 was located across the Space Station aisle on top of the Element 
Control Workstation described below. Two more cameras were located at the Ground Ex- 
pert’s area. 

Figure 2 


Video Hardware and General Equipment Layout 


SECOND FLOOR 
CONTROL ROOM 


HIGH BAY 


Entry 

Door 


Two Operators 
A AB 



Notes: 

1, Cameras Cl - C4 provide dynamic, 

real-time images lo the Maun TV 
monitor at the Control Room via the { 
Quad -Splitter 

2. Camera C5 connected to the 9" color 

monitor inside Glove Boa. 


DatefTume (0.1 sec) 
signal inserted on 
all TV images 

Video Tape Recorder 
(Panasonic 8050) 
QmmdeSplilter 
(Model 1290B) 

Afaia TV Monster 
(NEC 26", Model PR2600A) 

TV Camera C5 (NEC NC-8 
color, CCD, Aulo-iru, 28mm 
efl lens) (on Tripod) 

Expe ri m ente r's Table 
This table to represent a 
remote Pi. at University 

C5 output to color monitor 
insuic Glove Box 


ROOM 115 


Element Control 
Work Station 



TV Camera C4 inside 
Element Control 
Workstation Rack 


Glove Box 


TV Cameras Cl and C2 are the hori- 
zontal and down-looking cameras 
inside the Glove Box (not ^hown) 

TV Camera C3 is the miniature color 
camera mounted on robot arm. (not 
shown) 


Figure 3 

Audio Hardware and General Equipment Layout 


CONTROL ROOM 
SECOND FLOOR 


HIGH BAY 



Channel l ■ AU participants 
am same duplex circuit. 


Channel 2 ■ Ground Control 
Personnel and Ground 
Expert in separate 
room (excluding 
test subject ) 


PI 


Thi* table represent^ 
a remote expert's 
work area. 


ROOM 115 


o 

CREW C 

DJ 


Element Control 
Work Station 

The Flight crew- 
person will weir a 
belt-pack unil. 


N 


Glove Box 


Notes: 

1. All participants to have Hantrunics 

type headsets 

2. Each staouo lo have separate input 

volume control 

3. AU verbalizations to be tape- 

reconied on videotape 
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Figure 4 



The large format TV monitor displayed digital date and time (one second accuracy) and its 
i mag ery was recorded continuously on a timelapse VHS video cassette recorder (Javelin, 
model XL 1000) for later analysis. The reason for using a large format quad- image display 
and a second 19 inch single scene monitor (showing the entire glovebox from across the 
aisle) was to provide the ground controller with enhanced situational awareness of all activi- 
ties within and around the glove box and to be better able to evaluate what are the most ef- 
fective camera views to monitor all aspects of the operations. A video switching network 
was used which allowed the image from any camera to be displayed on any monitor. 

2. Glovebox. The glovebox consisted of a vertically standing, plywood enclosed, alumi- 
num frame (Space Station double-wide rack) structure with a transparent upper front sec- 
tion. During normal operation the user inserts sealed containers into the lower "entry vesti- 
bule", closes the front door, and then slides the "inner vestibule door'" open to remove the 
container or its contents. Figure 5 shows this structure. Its front face is 71.5 inches high 
by 39.5 inches wide. Its overall depth is 36 inches. The glovebox contained a number of 
items which included: (1) Macintosh computer with mouse input control (i.e., no keyboard 
was used), (2) high resolution 9" color TV monitor (NEC. model PM-971A). (3) program- 
mable robot (SCORBOT), (4) miniature color CCD TV camera (Toshiba, Model IK-M30) 
attached to the robot’s wrist, (5) two tube, 20-watt, ceiling-mounted cool white fluorescent 
fixture, and (6) front panel power controls. 
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Figure 5 



Three small color TV cameras were mounted inside the glovebox. Two were Panasonic 
model WV-CD-132 CCD cameras with 8 mm f 1 : 1 .4 fixed focus lenses. One of these was 
mounted above the ceiling of the glovebox aimed vertically downward with its optical axis ap- 
proximately centered on the inside floor of the glovebox. The second was mounted outside 
the glovebox on the right wall and aimed horizontally at the inside left wall. Its optical axis 
was approximately five inches above the floor of the glovebox so that it imaged a conical re- 
gion measuring about 26 inches diameter on the left wall and covering most of the floor. The 
depth of field of both cameras was from approximately 12 to 25 inches. The third CCD color 
camera was a Toshiba model IK-M30A with 7.5 mm fl: 1.6 fixed focus lens. This ultra-min- 
iature camera was rigidly attached to the jaw support wrist bracket of the robot so that it 
would capture the pincer motion and X-Y-Y motions without introducing any visual field ro- 
tations which were found to be highly disorienting. Its conical field of view subtended a half- 
angle of approximately 13 degrees arc. 

The lower right front of the glovebox contained a power control panel with the following 
(PRESS-ON, PRESS-OFF) lighted switches (top to bottom): Lights, .Air Supply. Vacuum, 

Downlooking Camera, Sidelooking Camera. Robot .Arm Camera, Main Power, CRT Power. 
Robot Power, Computer Power. Spare. Spare. The power-up/check-out procedure required 
the subject to activate certain of these switches at various times. 


RLACSTR 89.31 


Telescience Testbed 


Haines, et al. 


Page 8 


Located directly across the Space Station center aisle was another rack referred to as 
the Element Control Workstation (Cohen, 1988). A Panasonic model WV-CD110A color 
VT camera with wide angle 10 mm f 1:1.6 Switar fixed focus lens was installed so that it- 
pointed at the glovebox assembly. This image was displayed continuously in the con- 
trol/monitoring area to provide the controller with situational awareness within the entire ar- 
ea surrounding the glovebox. 

For protocol A, the robot was programmed to cany out a series of autonomous macro- 
movements; each one was a movement related to fluid transfer in microgravity, e.g., (a) 
grasp a syringe located in a holding fixture at location X, (b) rotate/unlock syringe, (c) with- 
draw syringe from fixture, (d) translate syringe to a second Hoaglands solution dispensing 
fixture, (e) insert, rotate, and lock syringe in place, (f) fill syringe with (imaginary) fluid, etc. 
The participant’s task was merely to monitor these movements and ensure that they were 
conducted properly and safely. Details of this software and hardware will be presented in a 
separate paper. 

3. Ground Expert Area. The ground expert represented a person at a university who 
was integrally involved in the flight experiment (e.g., as its principal investigator). One of 
his duties was to provide, step-by-step instructions to the flight crewperson in order to carry 
out the required tasks. This individual also provided trouble-shooting assistance in the 
event of hardware failure or clarification in ambiguous situations. The expert completed the 
workload ratings after each scenario along with the subject. This ground expert’s work area 
is shown in Figure 6. 

Referring to Figure 6, two 21" medium resolution color monitors were placed side by side 
(to help reduce total eye scan distance). A Mac II microcomputer (right side) presented the 
procedural steps which all parties followed. Its keyboard was not used during data collec- 
tion. An intercom with head-mounted microphone was also used by all participants. A clear 
flat workspace measuring 36" wide by 16" deep was maintained directly in front of the ground 
expert. Aimed down at the workspace was a readily moveable, miniature, color TV camera 
(Toshiba, model IK-M30A). Its imaged could be displayed on any monitor. A second CCD 
color TV camera with wider field of view was installed above the entire ground expen area. 
It provided overall situational awareness for others. Its image was rotated to correspond to 
the vantage angle of the miniature camera so that no mental image rotation was necessary. 
This was found to aid all participants in relating the two images to one another rapidly. 

Procedure: 

Training. Each of the eight flight crewpersons was carefully trained in the correct conduct 
of the three experimental science protocols during their first four hour-long training period. 
Each became familiar with ail equipment so that he or she felt comfortable with the displays 
and controls. Carefully planned motivational procedures became an important consideration. 
While a sufficient number of repeated sessions had to be run to ensure that asymptotic per- 
formance would be achieved, there was the possibility that loss of interest/boredom would 
ensue so that by the time the last ’critical” run was accomplished performance would no 
longer be asymptotic. The following motivational techniques were employed to try to com- 
pensate for this unwanted effect. 
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Figure 6 

Photograph of Ground Expert’s Work Area 



First, each participating mission specialist was treated individually and as a professional. 
The importance of this approach cannot be overemphasized. Indeed, each participant was a 
scientist or an engineer at NASA Ames. Each was expected to have a basic understanding 
of the overall test objectives as described in written materials provided to them before test- 
ing began. Second, each was trained by the same ground support team, i.e., ground person- 
nel were never switched throughout data collection. This helped to strengthen the interper- 
sonal ties and friendships which tend to develop during such testing. Indeed, in many cases 
the volunteer and ground communicator were on first-name basis as occurs in NASA flights. 
Third, each volunteer’s performance was monitored closely so that, if there was any indica- 
tion of changes, verbal encouragements could be given immediately and naturally. 

It was important during training to emphasize the importance of staying on an approxi- 
mate time schedule in order to make the automatically timed procedural steps more consis- 
tent from run to run and to maximize the amount of data collected. Each participant was told. 
-It i s very important for you to carry out each procedural step in a timely way. Do not waste 
time or wait excessively long between steps. 

Testing Schedule. Each participant spent two separate four hour-long sessions separat- 
ed by one day. An AM session participant was always retested on his second test day in 
the AM (similarly for a PM subject) to help reduce circadian-induced performance effects. 
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Testing occurred four days during each week, allowing the fifth day for maintenance, data col- 
lation, and analysis. Specific procedural events administered to each participant are de- 
scribed next. 

The participant was greeted and asked to read and sign an informed consent form 
(Appendix B). He was then given a verbal explanation of the study, its goals and objectives, 
the apparatus to be used, and provided an opportunity to ask questions. This was followed 
by actual hands-on demonstrations of: the audio communication system, the video camera 
and glovebox power controls, insertion of the hands into the robber gloves, use of the com- 
puter mouse control inside the glovebox and computerized subjective workload rating proce- 
dures, the stowage .compartment contents and related procedures. Science protocol practice 
sessions followed in groups of three. The next activity was a real-time demonstration of the 
three protocols in the order (A), (B), (C). Each protocol required a different amount of time 
and different skills. Appendix C presents the individual steps of each protocol. Protocol (A) 
refers to "Seed Germinator Charging ", (B) refers to " Plant Sample Acquisition and Preserva- 
tion," and (C) to "Remote Plant Observation.” 

During the first day’s training session each protocol was practiced twice as shown in Ta- 
ble 1. Four participants received no video mode at all, i.e., the color monitor inside the glove- 
box was turned off throughout all familiarization, training, waim-ups, and data collection 
runs. Their performance was compared with the performance of the four subjects who did 
have this form of telecommunications information. In addition, the monitors at the ground 
control and PI sites were also turned off in order to evaluate their contribution to the Pi’s and 
ground controller’s ability to carry out the required operations. 

Since protocol (A) required about 9 minutes to complete, (B) required about 18 minutes, 
and (C) required about 9 minutes, after training, a total of three groups of the three protocols 
(i.e., nine total) were completed by the end of the second day. The first group served as gen- 
eral familiarization and warm-up even though data was collected. This approach was taken 
in order to try to get the participant’s performance level at or near asymptote as quickly as 

possible. 

The second test session began at the same start time as the first test session but one 
day later. Participant 1 was then given protocols: (A)(B)(C) (cf. Table 1); followed immedi- 
ately by protocols (B)(C)(A) and then (C)(A)(B) with a ten-minute rest period separating 
each group. During the third and final group of protocols, (for example (C)(A)(B) for subject 
1 ) one of four carefully pre-planned telescience failures took place. They are discussed in de- 
tail below. 

Unique Failure Runs. A unique and different type of telescience mode failure was insert- 
ed unobtrusively by an experimenter during the last scenario to each subject (cf. Table 1, last 
bold-faced letter in each row); the flight crewperson did not realize it would happen. 
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Table 1 


Test Scenario Presentation Order 


COMPUTER IN 
CLOVEBOX 




VIDEO 

Training Sessions 




Data Collection Sessions 


1 


ON 

P-A-B-C 

1 

A-C-B 

2 

A-B-C 

3 

B-C-A 

4 

C-A-B 

5 

2 


OFF 

B-A-C % 

C-B-A 

7 

C-B-A 

t 

A-C-B 

9 

B-A-C 

10 

3 

ON 

ON 

C-A-B 

It 

C-B-A 

12 

A-B-C 

13 

B-C-A 

14 

C-A-B 

13 

B 


OFF 

A-C-B 

1C 

B-C-A 

IT 

C-A-B 

11 

A-B-C 

19 

B-C-A 

30 

5 


ON 

C-B-A 

21 

C-A-B 

22 

C-B-A 

23 

B-A-C 

34 

A-C-B 

23 

6 


OFF 

A-B-C ^ 

C-A-B 

27 

B-C-A 

» 

C-A-B 

29 

A-B-C 

30 

7 

OFF 

ON 

B-C-A 

31 

B-A-C 

32 

B-A-C 

33 

C-B-A 

34 

A-C-B 

33 

8 


OFF 

B-A-C 

36 

C-B-A 

37 

B-A-C 

3i 

A-C-B 

39 

C-B-A 

40 


Unique Failure Run Key; V- Video, A - Audio, W - Wotkjution, C - Collaboration 


Unique 

Failure 

Run 


V 

A 

W 

C 

V 

A 

V 

c 


(1) Video Failure. During this type of failure the ground controller announced that, "a pow- 
er failure had caused the TV monitor inside the glovebox to fail. All operations should be con- 
tinued on a "best performance basis". The failure always occurred just prior to conduct of pro- 
tocol (B) "Plant Sample Acquisition and Preservation." Likewise, all video downlinks from 
the glovebox to the ground expert PI also were turned off so that only a voice link and digital 
(computer screen) link remained. 

Of primary interest was the impact this failure would have on the ability of all partici- 
pants: (a) to successfully conduct the protocol, (b) What kind of response(s) was made? (c) 
How long did it take to reach a decision for each of the two groups (i.e., the four subjects pre- 
viously trained to carry out the scenario without video support versus the other four who had 
trained using video), and (d) on their workload ratings. 

(2) Audio Failure. When this type of failure occurred it was always during the last sce- 
nario. Ground control did not announce that a failure would occur. This event occurred about 
30 seconds into protocol (C) "Remote Plant Diagnosis" and the flight crewperson had to 
adopt alternative methods of communicating, terminate the protocol, or continue on an unau- 
thorized basis. When the experimental design permitted its use, all parties could use an elec- 
tronic mail system to communicate if they desired If a subject did communicate using the 
electronic mail and asked for instructions he or she was told, "we have had an audio system 
failure, proceed on a "best effort" basis." 
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Of particular interest was: (a) what type of response(s) were made, (b) how long it 
took to execute it (them), (c) the adequacy of the response, (d) were other available tele- 
science modes used? (e) the number and type of performance errors this failure produced, 
and (f) the impact on their workload ratings. 

(3) Workstation Presentation Failure. This failure consisted of unexpectedly present- 
ing garbled screen information for procedural steps 12 and 16 of protocol B Plant Sample 
Acquisition and Preservation" since it had the largest number of individual steps. It was rea- 
soned that the participant would have most difficulty remembering all of these steps and 
would tend to rely more on the screen display as a step-by-step cue. It always occurred dur- 
ing the last scenario of the experiment. 

This failure always occurred in step 12 which was supposed to state (on the monitor 
screen): "Position all items so that the inner airlock door can be opened", and again about 
one minute later on step 16 which was supposed to state, "Wipe down the glovebox with 
wet wipes and bag for trash." It was reasoned that these steps had been practiced many 
times before (i.e., during previous practice and data runs) and the participant may simply 
know what to do without prompting. On the other hand, the participant may wait for ground 
control to give another (ungarbled) go-ahead for this step. These subjects never waited for 
a go-ahead during this failure. 

When the "critical" step appeared in unintelligible form (i.e., random letters) on the screen 
it was of interest to find out: (a) what initial response was made, (b) how long did it take to 
find an acceptable solution to the problem (e.g., by asking the ground controller to read it to 
them, to recall it from memory and have ground control verify its accuracy), (c) did they use 
another available telescience mode to work around the error, (d) the number and types of er- 
rors made, (e) what unanticipated problems were encountered, (f) task accomplishment 
time, and (g) the impact on their workload ratings. 

(4) Collaboration Failure. This failure was produced by having the ground controller sim- 
ply not respond to the flight crewperson for a period of three (3) minutes. It always occurred 
during the last scenario and at a time when the ground controller should have been available, 
viz the start of step 10 of protocol A "Seed Germinator Charging." This step prompts 
the flight crewperson to " Click SEND to send a message to the POCC to prepare for monitor- 
ing Advance only after the POCC acknowledges the message." It was of interest to find out 
(a) how long the participant would wait for the ground support person to return, (b) whether 
they would simply ignore the lack of ground response and go ahead anyway, or take some 
other action (e.g., wait out the three minute-long period), (c) whether any (other) available 
telescience mode would be used and when, (d) the number and type of performance errors 
this event caused, (e) whether unanticipated problems would be caused by this failure, and 
(f) the impact on their workload rating. 

If the crewperson challenged the ground controller about what happened . the follow- 
ing response was always given after the failure period. "Were sorry (name of participant), 
we had a temporary communication glitch down here... hope it didn't get in your way. ' 
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Test Participants: 

Eight volunteers took part as surrogate spaceflight mission specialists. Four were male 
and four female. Their average age was 33.6 years. All were in good health and possessed 
20:20 corrected or uncorrected vision. None was familiar with the three protocols. Three 
were biologists with some familiarity with a glovebox (bio-isolation chamber) and the proce- 
dures that are involved in using one. Four were engineers and one was a computer science 
major. This diversity of background helped us to assess what is involved in conducting crew 
coaching and training to people with different disciplinary backgrounds. 

Since the basic objective of this test was to evaluate selected aspects of the telescience 
support system, the two experimental support team members were also considered as test 
subjects. The ground controller served as video system switcher when the PI or flight crew 
person requested different camera views. He also coordinated the test runs to ensure that 
the correct test schedule was followed and monitored overall quality of science being per- 
formed. The ground expert [also referred to as the principal investigator (PI)] played the part 
of a university scientist whose experiment was being carried out in space. He gave visual 
and voice coaching and prompts as needed. 

Two persons acted as expert raters throughout data collection. Each monitored the per- 
formance of the test subject as well as the two support team members. One rater had prima- 
ry responsibility for monitoring the hardware and software operations during the data collec- 
tion. The other rater observed the behavior of the subject, ground controller, and PI and also 
watched for system inadequacies. 


RESULTS 

The results are presented in the following sections: (1) Impact of Telescience on J^°* 
ductivity (2) Impact of Telescience on Task Accomplishment Time, (3) Impact of Tele- 
science on Dealing With Errors, (4) Capability of Using Telescience to Cope with Off Nomi- 
nal Events, (5) Glovebox Hardware/Software Redesign Comments, and (6) General Com- 
ments. 

In general, this experiment explored the general usefulness of three telescience modesof 
information presentation upon remote coaching of fairly complex scientific procedures. The 
ability to show the subject exactly how to carry out these procedures was significantly 
enhanced by having a high resolution color monitor located where it could be seen at all tunes 
(inside the glove box) and simultaneous audio transmission. Nevertheless, the audio chan- 
nel was considered to be the most critically important channel of the three tested. As is not- 
ed below, some subjects disregarded the computer-driven procedures checklist during audio- 

visual coaching. 

( l) Impact of Telescience on Productivity 
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Savings in Crew Time Results 

The following comments were made by the subjects during a group debriefing after the 
study. With regard to the workstation’s "electronic" check list one subject said, "I paid at- 
tention to the check list only the first time. Later I ignored it. I just went through the motions 
(to satisfy the requirements of the experimenter). For example, I checked off the seed germi- 
nator even though it was still attached to the side wall of the glove box. Another subject re- 
marked, "I liked the check list and would want to have a verbal command capability (to acti- 
vate each item). To be able to step through it would permit me to switch off one level of con- 
sciousness." Another subject said, "I liked the video (inside the glove box) at first but later 
I found it constraining, like a Big Brother type thing." Another subject felt that the experi- 
menters, who were also following identical check lists slowed him down in accomplishing the 
required tasks. The ground controller relied on the checklist throughout the entire study 
since he was constantly being distracted by outside, ongoing requests for assistance which 
drew his attention away from the ongoing science procedures. Having the real time checklist 
always available was a big help to him. The PI remarked, "My check list on the monitor may 
have slowed me down but it was really a help." 

Savings in Ground Controller and PI Time 

Having a video image to show exactly what the subject was doing during these science 
procedures proved to be extremely important. The ground controller estimated that he saved 
at least five minutes per procedure by not having to look up printed procedural steps in a 
manual. For the fifteen different scenarios completed by each subject this savings adds up to 
one hour and fifteen minutes. The PI did not have specific comments on this subject. 

Quality of Science Rating Results 


In general, the availability of audio, visual, and computer workstation information resulted 
in improved science quality. However, individual differences among these subjects qualified 
this conclusion. Some subjects with a biology background did not rely as heavily on the 
available telescience modes as did subjects who were not familiar with the hardware and 
procedures. In other words the telescience support helped compensate for deficiencies in a 
subject’s science background. 

Referring to Table 1, the first four subjects were trained with and always had video imag- 
ery available inside the glove box. This high resolution color view could be switched from 
one camera to another upon (verbal) command to the POCC. These four subjects tended to 
perform required tasks with greater precision and "rigidity", i.e., their performance tended to 
be better than the four subjects who had never had the benefit of video imagery. Both the PI 
and expert raters agreed on this. This tended to be true even for those subjects with very 
positive attitudes toward the experiment. As might be expected, those subjects with prior 
biological research expenence performed the tasks more precisely than did the non-biolo- 
gists despite the array of telescience support hardware and software available. Only the ex- 
pert raters could assess quality of science during tests on the last four subjects since video 
was not available to the PI or to the ground controller. The expen raters indicated that vid- 
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eo definitely contributed to higher quality of science because: (1) The PI could immediately 
assist the subject in preventing incorrect actions. An example of this occurred during a no 
video run when a non-biologist subject literally crushed a plant root into a cryo-stub pnor to 
snap freezing. If this had happened in space the PI would not have known about the turned 
sample until it was returned to Earth, (2) The PI could physically demonstrate a^w task 
from one or more visual vantage points for a subject prior to its accomplishment, (3) The Pi 
could tell when a procedure was over and could move immediately to the next task wi out 
the subject having to inform him that it was over, and (4) The subject was less free to exer- 
cise personal freedom in how a procedure was carried out when he or she knew the proce- 
dure was being watched by others who might correct them in public. 

The following comments were made by the subjects during the post-test debriefing: (1) 
"The video was very important during training. I liked to be able to see the PI s ace 
(personal identity and expressions), however, the actual flight crew may not like the rear 
view camera (located at the element control workstation rack opposite the glove box) yet it 
may lead to better science," (2) "Let the crew understand that they are part of a team and 
give them mutual respect and support Get a dialog going with them," (3) One of the engi- 
neer test subjects said, "How should we decide which (plant) sample to keep and which to 
throw out? The video led to my achieving better quality of science," (4) Another subject 
commented, ”...I verbalized each step out loud (while reading it off the glove box monitor), 
and (5) The PI felt that he wanted "...all available telescience modes available all of the 

time." 

The PI stated that, "The most important factor that influenced the quality of science was 
the two-way video. The video was essential for procedure verification, coaching by a ground 
expert, and coping with off nominal events." 

He continued, "Many of the procedures that the crew must carry out involve preservation 
of plant and animal tissues. Many of these techniques are highly specialized for a given dis- 
cipline and are very difficult to complete correctly. They are all or nothing propositions. If a 
small mistake is made the sample is lost. Having a ground expert observe the procedure 
from many angles made it possible for mistakes to be caught while there was still time to do 
the procedure correctly. During parts of the test where the video was shut off, the narrative 
from the crew indicated that the procedure was being completed correctly, but after review of 
the video tapes it was found that many mistakes were made that would have resulted in the 
loss of the experiment. During the course of a 180 day flight, many off nominal events 
(probably) will occur that the crew will have to cope with. They cannot be trained to cope 
with every contingency of every life science (or other) discipline. Video coaching will allow a 
discipline expert, who best knows the objective of the experiment, to cope with the off nomi- 
nal event " Still another consideration is that of shared responsibility; the present type o 
telescience support for mutual, real time collaboration essentially distributes the Lability for 
failed science procedures more broadly (and evenly) among all participants. 

The PI also remarked after the study that the greater the video exposure a subject had the 
more confidence he or she seemed to have toward the accomplishment of a given science pro- 
cedure during an off nominal run. 
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Subjective Workload Rating Results 

A total of 1020 separate workload ratings were made in this study, 480 by the ground 
controller, 480 by the PI, and 60 by each of the eight subjects. Two-way mixed model analy- 
ses of variance were performed with video on/off as one dimension and workstation on/off as 
the second for the three workload measures used as well as the three test scenarios. These 
findings across all test subjects are presented in Table 2. 

Table 2 

Statistically Significant Analysis of Variance 
Results for Mean Subjective Workload Results 
(P value cited) 


Workload Scale Source of Error 

Workstation Video Workstation x Video 

Test Procedure (On/Off) (On/Off) Interaction 

"Z 0.0006 

_ 0.0001 

0.0001 

0.044 0.007 

0.015 

0.013 0.001 

0.048 0.0004 

0.002 
0.002 


Mental A 

B 
C 

Physical A 

B 
C 

Total A 

B 
C 


When plotted, these significant two-way interactions for physical effort were the result of 
the fact that mean workload decreased from the video off to video on condition when the 
workstation was on and increased when the workstation was off. This finding not only sug- 
gests that the two telescience information sources complement one another by decreasing 
physical workload when both are on but also that the video information played a more power- 
ful role than did the workstation. 


Flexibility of Scheduling Results 

It was discovered that the present three node telescience support system did not contrib 
ute significantly to an increase in scheduling flexibility. This may be due to the fact that the 
experimental design called for a rigid activity schedule by all three participant groups. A sub- 
ject remarked during his post-test debriefing that, "he would use on-board video tapes of 
planned procedures and supplement them with ad lib coaching where necessary. 
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Transaction Management Results 

The management of all science related transactions will assume increasing importance as 
the number of experiments to be carried out increases and the time available to conduct them 
diminishes. The proper use of telescience related procedures and hardware should positively 
impact this management role. 

It was found that the video off test condition was very boring for both the PI and the 
ground controller and tended to lead to increased environmental distractions and missed pro- 
cedural steps. In general, the fewer number of procedural steps to carry out the faster the 
subject got bored. More errors tended to made as boredom level increased. The level of ver- 
bal interchange between the subject and the PI and ground controller also increased. A sub- 
ject remarked, "It is far fetched to anticipate on-orbit astronaut training, particularly with the 
limited bandwidth that will probably exist to and from Earth... (such training) will pose a sig- 
nificant procedural problem." 


(2) Impact of Telescience on Task Accomplishment Time 
Time Line Analysis Results 

A computer logged the elapsed time of each mouse click made by all three participants 
(subjects, ground controller, PI). This was done to obtain data on typical time to accomplish 
the various tasks; task performance time is useful in planning for training, etc. Figures 
present total time to accomplish scenarios A - C, respectively by each subject. The filled cir- 
cles are for the unique failure run which always resulted in the longest accomplishment times. 

Procedure elapsed time averaged across the three scenarios and all eight subjects ranged 
from 7.3 to 27.1 minutes for the first data collection period on the test day, from 4.6 to 23.6 
minutes for the second data collection period, and from 5.8 to 25.2 minutes for third and final 
data collection period. There was no pronounced reduction in task accomplishment time sug- 
gesting that these subjects had already reached stable task accomplishment times by the 
test day. This may also support the general observation that these subjects required only 
one training day to attain relatively stable task accomplishment time and proficiency. 

The electronic checklist inside the glove box proved to be a useful and popular source of 
information for the subject. Having future required procedural steps visible made it possible 
to cany them out in a more relaxed, flexible, and continuous manner. In this regard a subject 
commented, "I found myself checking and carrying out present and future activity steps and 
then clicking the mouse to indicate I had completed them as the instructions required. 
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Figure 7 

Performance Elapsed Time by Subject 
for Scenario A 



Figure 8 


Performance Elapsed Time by Subject 
for Scenario B 
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Figure 9 

Performance Elapsed Time by Subject 
for Scenario C 
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Use of an automated checklist displayed inside the glove box may be more useful when 
one is learning new procedures than it would be later, particularly for simple experimental 
protocols. Most of the subjects learned the procedures so thoroughly by the second day that 
they didn’t have to refer to the checklist very often and several claimed that it slowed them 
down. Using "flexible software" that can be readily modified over the course of a mission to 
match the current (and changing) requirements of each subject may well be a better approach 
and should be evaluated experimentally. 

Several subjects remarked about the presence and use of a rudimentary robot inside the 
glove box. It must be noted that this study was not designed to evaluate robotic operations; 
the robot and control system was included to demonstrate several basic fluid handling proce- 
dures that had been developed earlier (Schooley et al., 1989). During the post-test debrief- 
ing one test subject said, "The robot seemed like a useless procedure, i.e., doing a variety of 
tasks OK rather than one task perfectly." Another subject remarked, I would rather be my 
own robot on limited numbers of repetitive procedures." 


The ^round controller found the computer check list to be very useful in keeping him in 
step with die ground controller and PI. However, when the subject forgot to click on an item 
the controller's workload increased since he had to remind the subject to make the necessary 
response(s). This happened about five percent of the time. 
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Total mean elapsed time to complete scenario B with video on was 18.7 minutes and only 
13.5 minutes with video off (across both workstation states) (F = 12; df = 1; P = 0.002). 
While the video on condition tended to slow down procedure accomplishment time the overall 
quality of science carried out was far higher (see discussions in sections 1 and 3). 


(3) Impact of Telescience on Dealing With Errors 
Error Analysis Results 

The potential always exists for human errors to be made during long, complex, and repeti- 
tive manned procedures (Meister, 1982). There are errors of omission and commission 
which range in severity from innocuous to disasterous. A number of types of errors were 
identified in this experiment which illustrated the importance of having a television monitor- 
ing capability for ground-based personnel to monitor crew performance remotely. Having a 
communication link between the ground and on-orbit personnel made it possible to monitor 
performance and behavior more consistently. This is illustrated by a comment made by the 
PI after the test. He said, ’’The crew said they were doing the procedure one way but were 
actually doing it another. My TV gave me immediate insights about this." 

The PI continued, "The loss of video resulted in many errors committed by the crew that 
were not caught by the ground PI. Most of these errors would have resulted in the loss of 
the tissue sample, and some would have resulted in a safety hazard and/or injury to the 
crew. Undetected errors included poisonous glutaraldehyde fixative spills, broken bioisola- 
tion between the crew and specimens, frostbite of a crew’s fingers, too much tune between 
tissue removal and freezing and fixation, mechanical damage of samples, incorrect cryo 
freezer operation, incorrect positioning of a plant for an infrared scan, and incorrect descrip- 
tions of plant pathology. As has been described above, the video view for the ground expert 
is necessary to catch mistakes as they happen so that the procedure can be repeated if nee- 
essary. Audio narrative by the crew made it sound like everything was going fine when it 
was not. The loss of the electronic checklist also caused the crew to skip steps and to lose 
their place in the experiment several times." 

A detailed analysis of the post-test video tapes showed that there were sixteen major 
procedural errors committed across the four subjects to whom video was not available during 
their 5.4 hours of total testing. There is every reason to believe that most or all of these er- 
rors would have been caught if real time video coverage had been available. 

The expert raters commented on a number of errors which took place. They noted: (1) In- 
advertent activation of audio buttons and screen (icon) burtons. Both the ground controller 
and PI made incorrect manual responses to screen icon burtons when the subject did not re- 
spond quickly enough. It was as if they were trying to respond in place of the subject to keep 
the experimental procedures moving along. (2) On several occasions the macintosh sort- 
ware failed to proceed through the full list of checklist items and had to be rebooted. (3) 
When the printer was operating it inhibited use of the macintosh. This delayed the start ot a 
scenario several times by about five minutes. (4) Sometimes several subjects would finish 
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two or more consecutive steps in a scenario before clicking the mouse on them. This was 
done primarily during the use of equipment gathering and stowage checklists where many 
items had to be identified and moved to another location. The subject was told th y 

could click off all of the items after all of the items had been correctly moved. While g 

their tasks smoother, this kind of response "error" made the automated timing ofeach suc- 
cessive step in error to some degree. In several instances two subjects clicked off aU items 
during a post-test stowage activity even though a major piece of equipment was still inside 
the glovebox and in full view! Since it was velcro-attached to a side wall and not lying on the 
bottom surface of the glove box with the rest of the equipment, it was overlooked. (5) The 
on-screen command buttons to activate the robot in scenario A were sometimes used incor- 
rectly leading to lost time when the software had to be reinitialized. Later in the experunent 
when this occurred the team simply went on, disregarding the consequences. (6) e 
ground controller inadvertendy left the microphone on several times while he discussed ex- 
perimental details with the PI and others present. The subject overheard this conversation 
with unknown consequences. A "hot mike" panel light would have helped reduce this possi- 
bility. (7) On several occasions the subject did not wait for the proper acknowledgement be- 
fore initiating the robot motion in scenario A. 


Unanticipated Problems 

A number of unanticipated problems occured during this experiment. They included: (1) 
Differences in teiminology/names used by all three participants for some of the objects use 
to carry out a scientific procedure. For example, the inner glove box door was actually a hor- 
izontally sliding surface and was confused on occasion with the outer vestibule door. This 
kind of confusion was infrequent, however. At times a subject would merely point at some- 
thing to identify it knowing that someone probably would be watching and could identify it. 
(2) A TV camera was incorrectly aimed during one run so that a desired scene was not 
achieved. This problem probably arose from an inadvertent bump of the camera between 
tests and the fact that camera view checks were not made pnor to testing each day. (3) A 
time lag of approximately eight seconds occured (due to computer processing time) tween 
the time the subject made a mouse input to a check list and the time both the PI s and the 
ground controller’s computer screen accurately reflected then mouse input. This delay e o 
some coordination problems early in the study until the nature of the problem was deter- 
mined Thereafter appropriate allowances were made to resynchronize all participants. 
These usually took the form of the ground controller announcing over the intercom that there 
was a transmission delay and not to initiate a following step until all screens were synchro- 
nized The subject acknowledged this verbally. (4) A discussion held between the PI and 
around controller dealing with the interpretation of a particular instruction was overheard by 
die subject with unknown consequences. (5) As the subject became more and more familiar 
with the instructions he or she would carry out procedural steps without looking at the com- 
puter check list at all. In several instances the wrong procedure was earned out. ( ) ne 
subject accidentlv cut his finger with the scalpel. While there was almost no blood loss he 
did have to stop the procedure to deal with it. Future bioisolation facilities must take this 
kind of accident into account and provide for immediate, self-actualized first aid. W) During 
scenario B the subject accidently pulled the stopper out of the glutaraldehyde bottle. It it had 
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actually been filled, this highly caustic fluid would have spread throughout the glove box 
causing various problems (sample and equipment contamination, electrical shorting, TV cam- 
era lens coating, etc.)- The subject mentioned the spill almost immediately. Both the PI and 
the ground controller also saw it happen and were ready to verbally assist the subject in the 
clean-up procedures which had to be carried out. Training was not provided for such clean up 
procedures prior to testing. This accident slowed the procedure’s completion by about three 
minutes and demonstrated the importance of performing procedurally correct simulations with 
realistic hardware. 


(4) Capability of Using Telescience to Cope with Off-Nominal Events 
Crew Performance Results 

One of the objectives of this study was to assess the role played by video, audio, and 
computer workstations in coping with four different kinds of unexpected failures. Each type 
of failure is discussed separately. In general, when the communication system failed these 
subjects showed that they could still complete the tasks by turning to alternative telescience 

modes. 

Video System Failure. It will be recalled that the video (TV) display inside the glove box, 
at the POCC, and at the Pi’s station all failed simultaneously during protocol B. This proved 
to be both advantageous and detrimental to different subjects. Several questions were of in- 
terest: 

(1) What kind of responses were made? Interestingly, the PI had grown so accustomed to 
relying on the computerized checklist information that when it failed early in the study he did 
not have a hard-copy backup to refer to. He had to request assistance from the ground con- 
troller at this time which lengthened the time needed to accomplish the procedure. Neverthe- 
less, the procedure was completed successfully. Most of the first four subjects who were 
trained using the video system simply relied on their memory to complete the required tasks, 
i.e., the failure (probably) occurred too late to measure anything of real importance here. 
From the ground controller’s standpoint this failure did not significantly influence his task ac- 
complishment due to the high degree of overlearning that had taken place by the time this 

failure had taken place. 

(2) What type of responses were made? Subject 7 coped well. He simply progressed with 
the now familiar procedural steps and verbalized what he was doing to the PI. His high level 
of familiarity with the required steps made the loss of video a minor inconvenience at most. 
Most of the subjects waited for some clarification of the problem from the PI or POCC. 

(3) How was performance affected? In general, the unexpected system failure did not signifi- 
cantly influence task accomplishment. Since it always occurred on the last scenario the sub- 
jects were already trained (and also motivated) to complete all required tasks. 
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One subject remarked, "...when my video went out I could do it (the procedure) my own 
way." This subject was not a plant biologist. Another subject commented. It was a chal- 
lenge intellectually when I had no video... I had to visualize the steps... This helped to reduce 

boredom." 

Audio System Failure. Some questions raised here included: 

(1) What type of responses were made? In general, it was noted that the subjects were high- 
ly dependent upon the audio system. When it stopped working a variety of responses 
ensued. Subject 6 simply stopped and did nothing at all while subject 8 was very patient, ex- 
pecting the procedure to continue even though communication was lost. We think that be- 
cause the video system was less obtrusive than the audio, when the audio failed the sub- 
jects did not rely upon the video as much or as rapidly as they relied upon the audio when the 
video failed. This interpretation is only tentative and should be researched further. 

(2) How long did it take to carry out the required tasks? Task performance time did not 
lengthen noticeably during the audio failure condition as compared with the normal audio con- 
dition. 

(3) How adequate was the response? Most of the subjects simply continued on with their 
tasks despite the failure. 

(4) Were other telescience modes used? Several subjects tried to use hand signals over the 
TV system to communicate with the PI but soon gave up and went on with the task. 


Workstation Presentation Failure. This unique failure run only occurred for subject 3 who 
happened to be a biologist. While it is unwise to draw conclusions from a single subject sev- 
eral observations were made. Some questions of interest here included. 

(1) What initial response was made? This subject simply shrugged and went on with the 
required tasks. She also inquired about what had happened and asked whether it would be 
fixed? In general, failure of the workstation did not cause the subject much more than incon- 
venience. When she forgot a step she simply asked for help from the PI. That is, the audio 
visual channels were relied upon for task accomplishment more than the workstations. 

(2) How long did it take to find an acceptable solution to the problem? This subject contin- 
ued with the procedural steps almost immediately. 

(3) Was another available telescience mode used to correct the error? Yes, the audio chan- 
nel was used by the subject to inquire about the nature and duration of the problem and to 
request detailed procedural advice from the PI. 

(4) What unanticipated problems were encountered? None. 

(6) What was the task accomplishment time? This failure added 5.5 minutes to the amount 
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of time needed to complete this scenario. Reference to Figure 7 (cf. upper most data point for 
subject 3) shows that mean accomplishment time for the other two scenarios was about ). 
minutes while the failed workstation in scenario B required 14.6 minutes. By comparison, the 
ranee of task accomplishment times for the other seven subjects for this scenario (also car- 
ried out during the last data run) ranged from 10.4 to 20.8 minutes indicating that this sub- 
ject’s task accomplishment times were faster than the average to begin with. 

(7) What impact was there on workload ratings? None. This subject’s ratings did not change 
following this off-nominal run. 


Collaboration Failure. This failure consisted of the ground controller not responding verbal- 
ly to the subject for three minutes during scenario A (Seed Germinator Charging) at the point 
when the ground controller was supposed to authorize the subject to activate the robot s mo- 
tions. Several questions were of interest in this situation. They included: 

(1) How long did the subject wait for the ground support person to return? Subject four initi- 
ated the robot without authorization after waiting about one minute into the "silent period. 
The other subject (no. 8) waited the full three minutes and did not initiate the robot at aU. 
Personality and other individual differences seem to play a strong role here which available 
telescience communication modes may only modify slightly. 

(2) What response would the subject make? See previous paragraph for specific details. In 
addition, neither of the two subjects involved tried to communicate with the POCC using the 
electronic mail system at the Element Control Workstation. 

(3) What kinds of errors would be caused? None were noted, however, this may be because 
the robot was already pre-programmed to carry out a fixed series of operations which only 
needed to be initiated and then visually monitored. 

(4) What unanticipated problems would be caused? None. 

(5) What impact would this have on workload ratings? Neither subject’s physical, mental, or 
total workload ratings changed following this unique failure run from values given during pre- 
vious runs. 


(5) Glovebox Hardware/Software Redesign Comments 

While not a central aspect of the study, the present rapid prototyping activity did elicite a 
number of interesting comments regarding design improvements to the present 
mockup. The subjects' own comments are presented here in no particular order: (1 Those 
items which are used m most or all of the procedures should be located ms.de die glove box 
itself, for instance the trash bags and wet wipes ' (2) "The depth of the glovebox vest! u e 
was too great. As I reached down into it with my gloves my microphone kept hitting the 
front glass." (3) "I didn’t use the video very much on day 1 but (I) learned to appreciate it on 
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day 2. "I only needed the video in the beginning. Later I did ask for it to see down into t e 
vestibule." (4) Several subjects commented on the value of being able to click the e ec- 
tronic checklist by a voice input. A subject remarked. "It may be important to be able to tai- 
lor the use and design of the check list." (5) One subject suggested that all glove box equip- 
ment that had to be installed and later stowed should be represented on the computer screen 
by their oudine (shape). When they are clicked off (after being properly stowed) each out- 
line should disappear. (6) On numerous occasions subjects would leave the inner vestibule 
door ajar as they opened the outer door. An interlock system must be installed to prevent 
this from happening in space. (7) One subject used the tiny condenser microphone mounted 
to the front of the glove box to scratch her nose on. Another (female) subject commented 
that her hair tickled- her face when her hands were in the gloves and not available to deal 
with the situation. 

The presence of the five-axis robot inside the glovebox prompted a number of comments, 
e.e. (1) "A robot will be useful on repetitive tasks and in carrying out operations in harsh en- 
vironments.” (2) "The robot should do the snap freezing operations." (3) "The robot took up 
a large volume for (giving) relatively little value. The crew still would need to monitor its ac- 
tivities rather than having the POCC do the monitoring.” This comment raises the important 
issue of how properly designed telescience acts to redistribute the responsibility for opera- 
tions monitoring. This may not be good since there is the possibility that all participants will 
reduce the amount of their active, deliberate monitoring because they (may) assume that an- 
other (remote and thus unseen) participant is doing so. 


(6) General Comments 


An interesting comment made by an expert rater was that the various steps making up 
each test scenario are not of equal value from the standpoint of achieving high quality sci- 
ence. Indeed, some are far more important contributors to science quality than are otters. 
Should the relative importance of one step versus another in a science procedure be indicat- 
ed, particularly to the less experienced subject? 


Several lessons were learned regarding control switching from one participant to anod^ 
er (1) It was found to be very helpful to be able to switch TV camera views from the POCC 
rather than burden the subject or PI with this task. This switching was accomplished quickly 
(typically within several seconds after a request was made). It helped the PI to monitor the 
subject’s performance in a time-efficient manner and the subject to have useful video infor- 
mation available when needed. It also prevented the subject from having a different camera 
view inside the glove box than what was being displayed to the PI. i.e., both views could be 
pre-set to be the same by the TV image controller. (2) The POCC was much faster an 
made fewer switching errors when the PI used a standardized numbering system for the cam- 
eras and monitors. (3) The audio system used was full duplex between all three nodes. - 
ins the no video trials neither the PI nor the ground controller could see the subject when he 
finished a procedure; both had to rely on the audio channel for this information. This resulted 
in some confusion while passing control from one participant to another which a half-duplex 
system would have obviated. It was felt that a full video system would not require a half-du- 


RIACSTR 89.31 


Telescience Testbed 


Page 26 


Haines, et al. 


plex system because of the immediate visual confirmation of activities that is possible. 

This experiment demonstrated the value of being able to rapidly prototype different teleop- 
erations-related information systems. Basic screen icons were "drawn" quickly using hyper- 
card and evaluated from a human factors design standpoint prior to their inclusion m this 
study. (Johnson, in preparation) The location of displays and controls also could be varied lat 
will since full scale cardboard mockups were used before constructing the final glove box. 1 
camera fields of view, placement, and aiming were rapidly changed and evaluated before the 
testing began. In short, many valuable new insights were gained both before and during this 
experiment because of the inherent flexibility of the hardware and software. 


DISCUSSION 

This study has shown that carefully planned audio-visual and computer workstation 
hardware can be used effectively to accomplish various kinds of life science procedures re- 
motely and that relatively untrained subjects can learn new life science procedures under the 
careful guidance of a trained coach. It is important that all participants maintain an overaU 
situational awareness of what is going on at all times. This awareness permits each partici- 
pant to act in a more closely coordinated manner and to back up the other participants. Lost 
time due to misunderstandings and procedural errors also are reduced while morale seems to 
be enhanced. The fact that use of a computer check list slowed the more expenenced sub- 
jects in task accomplishment indicates a need for a more flexible software working environ- 
ment that can be tailored to fit the needs of different experience levels. 

There are still, however, a number of general concerns which should be raised with regard 
to the conduct of future teleoperations research. This list is not exhaustive. 

1. What is the best mix of manually operated and automated experiments? The 
answer may depend on whether or not there is a need to cany out many identical steps 
and/or whether the operation may be hazardous to the human. This study found that manu 
operations were cost and time effective when the procedures could be modified in real time 
the ground controller or the P.I. The P.I. remarked, "This reprogramming was instanta- 
neous." Humans should be permitted to do what they do best and the machines should be 
programmed to do what they do best. 

2. What quantity and quality of data is needed by the remote user? Clearly, each user 
has different requirements. Further research is needed to define what transmission delays 
are acceptable in the audio, visual, and digital data channels (Haines, 1989). 

3. What specific standardized data, human communications, and control interfaces and 
protocols are needed within specific scientific and operational disciplines to support the wide- 
ly varying needs of scientists? The present linked computers transferred alpha-numeric and 
Graphic data at 60 Hz update rate except when certain computationally intensive operations 
were underway, then screen update slowed markedly. The ground controller and P.I. had to 
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wait during these delays which effectively prevented them from accomplishing other tasks. 

The mouse was the only input device inside the glovebox used by the subject. All sub- 
jects found that it could be used effectively to click-off each procedural step. No training was 

required for its use. 

A high quality audio intercom was found to be an important component in this experi- 
ment. It had a frequency range of from about 200 to 12,000 Hz. and permitted correct discrim- 
ination of voice inflections. This system made it possible to identify the identity of the sub- 
ject entirely on the basis of their voice (and not video). 

4. What is the best mix of distributed (i.e., remote) versus centralized (i.e., payload op- 
erations control center) experimental control? Given that both locations have the same de- 
gree of data resolution, then either location should be permitted equivalent access to the data 
base. The final answer will depend on whether there is a need for data security or access 

control. 

5. What is the best way to decide how to allocate limited resources on Space Station to 
investigators with already approved experiments versus those who have more recently pro- 
posed "experiments of opportunity?’ 

6. What kind of command interlocks and data encryption are needed for critical opera- 
tions? 


SUMMARY AND CONCLUSIONS 

This study has provided a number of lessons learned concerning how three basic kinds of 
telescience support technology are used to support remote coaching of complex life sciences 
procedures. They include: (1) The capability to conduct rapid prototyping studies was 

shown to be a cost-and time-effective way to validate new telescience hardware and proce- 
dures to obtain empirical data, and to gain valuable hardware- and software-related in- 
sights (2) Hardware design will be influenced by the way the test procedures are written. 
(3) Properly designed audio-visual telescience support hardware will produce higher quality 
science output on Space Station Freedom. Whether or not computer-based workstations 
with procedural checklists may contribute significantly to productivity, error control, or work- 
load reduction remains to be seen. Further research is needed in this area. 
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APPENDICES 

Appendix A. Expen Observer Rating Form 


Group/Cell No. & Scenario Letter: 
Rater’s initials: 


Sheet No 


Subject Initials: 

Test Date: 

Time: 


1989 


Evidence for any of the following? 
a) Savings in crew time: 


Comments 


No 


N/A 


,Yes 


b) Scheduling flexibility improvement: 

c) Productivity enhancement: 


d) Science quality enhancement: 

e) Enhancement of Safety: 


Was a demonstration given of: 

f) Workstation teleconferencing (to assistn flight crew). 


g) Video teleconferencing (ditto): 

h) Audio teleconferencing (ditto): 

i) Effective information transfer. ■ — ■ 

Comments and observations regarding interactional telescience modes: 

j) Space to ground in general: 1 

k) Ground to space: 


l) Ground to ground: 

m) Robot and its control:: 
Number and type of errors: _ 


Unanticipated Problems: 
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Appendix B. Informed Consent Form. 


Exp. I -A 
11-15-88 


The studv in which vou are about to take pan is designed to evaluate the effectiveness of 
various telescience ooerating modes applied to a Space Station mission For instance 
video audio and digital communications links will provide you with key information needed 
to carrv out your tasks. You will be considered to be a Mission Specialist on Space Station 
whose "task it is to conduct three separate scientific protocols. You will be alone m the simu- 
lated Space Station mockup area. There will be a team of remotely located ground controllers 
present at all times. In addition, there will be a remotely located "ground expen available to 
assist you in certain required operations* 


Your involvement will require two, half-day sessions of four hours each. The fint half 
dav wiH be required for familiarization and protocol training. The second half-day will be 
u^d for actual data collection, once it has been determined that asymptotic pertormance has 
been achieved. We win try to accommodate your work schedule by your assignment to an 
AM or a PM session. Rest breaks are planned mid-session. You wiU be tested two ys 
apart, e.g., either a Monday - Wednesday or Tuesday - Thursday, We wiU begin the AM 
session prompdy at 0830 hours and the PM session prompdy at 1315 hours. All testing 
take place in room 115 of building 240A. It is important for you to stay within this area during 
your work period. Once testing is over a general tour may be given to you upon request. 

You win be provided with a NASA flight suit to be worn during data coUection. You will 
not need to bring any special equipment as everything wiU be provided to you. Feel ee to 
bring a camera for photos if you like. 


While every attempt has been made to make the testing environment as safe as possi- 
ble there is a smaU possibiHty that you may injure yourself. You should exercise caution in 
moving about within the testing area and in watching out for protruding structures electrical 
hazards, or other potentiaUy injurious conditions. Minor first aid will be available in the 
event you need it If you Simply explain the situation to your ground controller using the 
intercom system. You will be monitored continuously via a closed circuit TV camera sys- 
tem. By signing this form below you are recognizing this fact and agreeing to the recording of 
your voice (audio) and visual image (video) only for data analysis purposes. Under no cir- 
cumstances wfll these recordings be made public. In compliance with mandated privacy 
laws, every attempt wfll be made to code your data so that another person will not be able 

to identify you with it at a later date. 


j (print name) do hereby signify that I understand 

and agree to all of the above information and testing requirements. I acknowledge that I 
may leave this test at any time for any reason without prejudice shown towards me. 


Signature 


Date 
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Appendix C. 

Test Protocols (A) "Seed Germinator Charging , 

(B) "Plant Sample Acquisition and Prerservation," (C) "Remote 

Plant Observation." 

Key: C = flight crew, G = ground controller, PI = Principal investigator (ground expen) 

(A) Seed Germinator Charging 
C 

1. Close glovebox (GB) vestibule door. 

2. Open inner airlock door. 

3. Position all items from vestibule to top of inner airlock door. 

4. Close inner airlock door. 

5. Place several seeds in a germinator paper, fold, and then insert into seed germinator. 

6. Place and secure seed germinator within red outline for robot access. 

7. Place and secure syringe and holder within yellow outline for robot access. 

8. Place and secure distilled water container within blue outline for robot access. 

9. Place and secure Hoagland’s solution container within red outline for robot access. 

10. Click SEND to send a message to the POCC to prepare for monitoring. 

(Advance only after the POCC acknowledges the message) 

G 

1 1 . Click SEND to initiate robot fluid transfers. Immediately after the robot starts moving 

advance to the next step. 

12. Monitor robot extraction of distilled water. 

13. Monitor robot extraction of Hoagland’s solution. 

14. Monitor injection into germinator. 

15. Click SEND to send a message to the crew that the robot activity is complete. 

(Advance only after the crew acknowledges the message.) 

C 

16. Visually confirm that the robot completed the fluid transfers. 

17. Position all items so that the inner airlock door can be opened. 

18. Open inner airlock door. 

19. Move all items to GB vestibule. 

20. Close inner airlock door. 

End 


(B) Plant Sample Acquisition and Preservation 
C 

1. Close GB vestibule door. 

2. Open inner airlock door. 

3. Locate all items from vestibule to top of inner airlock door. 

4. Close inner airlock door. 

5. Open a specimen chamber and remove a single plant. 

6. Click SEND to request coaching from ground scientist. (Advance only after the PI 

acknowledges the message.) 
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7. Coach crew on tissue removal from apical meristem on root and shoot. 

8. Coach crew on tissue sectioning and preparation of two samples for snap freezing. 

9* Coach crew on preparation of two samples using glutaraldehyde for chemical 

preservation. 

10. Click SEND to inform crew that the coaching is concluded. (Advance only after the 

the crew acknowledges the message.) 

C 

1 1. Label both frozen and fixed sample containers. 

12. Position all items so that the inner airlock door can be opened. 

13. Open inner airlock door. 

14. Return all items, except for the wet wipes and baggies, to GB vestibule. 

15. Close inner airlock door. 

16. Wipe down the GB with wet wipes and bag for trash. 

17. Open inner airlock door and place trash in GB vestibule. 

18. Vacuum GB work volume and vestibule. 

19. Close inner airlock door. 

End 


(C) Remote Plant Observation 
C 

1. Close GB vestibule door. 

2. Open inner airlock door. 

3. Move plant chamber and baggies from vestibule to top of inner airlock door. 

4. Close inner airlock door. 

5. Remove plant specimen from plant chamber and replace lid. 

6. Place and secure leaf within green lined examination square for down looking camera 

observation. 

7. Click SEND to send message to ground expert to examine leaf color and morphology. 

(Advance only after the PI acknowledges the message.) 

PI 

8. Tell the crew approximately how long the diagnosis will take. 

9. Diagnose the plant for specimen health. 

10. Click SEND to send a message to the crew to reposition specimen. (Advance only 

after the crew acknowledges the message.) 

C 

11. Place and hold leaf on yellow intersection on GB wall for IR scan. 

12. Click SEND to send message to the ground expert to start IR scane. (Advance only 

after the PI acknowledges the message.) 

PI 

13. Perform the IR scan. 

14. Click SEND to send a message to the crew to reposition specimen. (Advance only 

after the crew acknowledges the message.) 

15 Coach crew how to hold specimin for optimal view of vernation. 

16. Consult with a NASA plant pathologist at the POCC to evaluate calcium levels 
in nutrient solution and daylength period.. 
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17. Instruct crew to adjust daylength setting on the habitat appropriately. 
C 

18. Adjust daylength setting if necessary. 

19. Insert plant chamber in plant habitat. 

20. Bag used cleaning materials for trash and place in GB vestibule. 

21. Open inner airlock door. 

22. Move all items to GB vestibule. 

23. Close inner airlock door. 

End. 
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